DOCUMENT RESUME 



ED 348 024 



IR 015 761 



AUTHOR 
TITLE 

PUB DATE 
NOTE 



PUB TYPE 



EDRS PRICE 
DESCRIPTORS 



Signer , Barbara R« 

A Study of Black At-Risk Urban Youth Using Computer 
Assisted Testing* 
Feb 92 

lip* ; ln< Proceedings of Selected Research and 
Development Presentations at the Convention of the 
Association for Educational Communications and 
Technology and Sponsored by the Research and Theory 
Division? see IR 015 706. 
Reports - Research/Technical (143) — 
Speeches/Conference Papers (150) 

KF01/PC01 Plus Postage* 

* Academic Achievement; 'Computer Assisted Testing; 
♦Diagnostic Teaching? Diagnostic Tests? *High Risk 
Students? High Schools? *Katheaatics instruction? 
Microcomputers? * Student Attitudes? Urban Areas 



ABSTRACT 

The objectives of this study were to develop, 
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Testing High-Risk Urban Students (MATH-R-US) . The project produced 
diagnostic software to meet the following criteria; (1) help students 
obtain high school mathematics credit needed for graduation? (2) 
motivate students to learn mathematics? (3) account for erratic 
student attendance? and (4) use computer adaptive testing as an 
integral part of the program* The project was used for an entire 
school year by a class ir an urban high school with an at-risk 
predominantly black population and a high rate of absenteeism. The 
tests, which accept generative responses rather than multiple choice 
answers, were administered once a week in the school's computer lab* 
The results of each test were saved and practice sheets, with answer 
keys, were generated for the missed objectives. The program was 
evaluated to improve implementation and furnish descriptive data to 
the classroom teachers and school administrators* it was found that 
the program generated intense student competition to see who could 
get the most hamburger graphics — which appeared on the screen when 
students completed a test with 100% accuracy — in an hour. Both males 
and female students expressed positive attitudes about the course 
components, but female responses reflected more confidence in their 
own abilities* Computer math test scores indicated a consistent 
improvement on rete sting of a topic, with 23 perfect scores on 43 
retests* A discussion of the implications of this study concludes the 
paper. (2 tables, 22 references) (BBM) 
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A Study of Black At-Risk Urban Youth Using Computer Assisted Testing 

Objectives 

The objectives of this study were to develop, implement and 
evaluate the year long project. Microcomputer Adaptive Testing High- 
Risk Urban Students (MATH-R-US). The project produced diagnostic 
software to meet the following criteria: (a) help students obtain high 
school mathematics credit needed for graduation, (b) motivate students 
to learn mathematics, (c) account for erratic student attendance, and (d) 
use computer adaptive testing as an integral part of the program. 

Theoretical Framework 
The theoretical framework of this study was based on the 
literature concerning: (a) computer adaptive testing of computational 
skills, (b) assessment as part of instruction, and (c) computer assisted 
instructionAesting with high risk students. Computer adaptive testing 
has been recognized as an effective means of assessing student 
performance of computational skills (Attisha & Yazdani, 1983; Janke & 
Pilkey, 1985; McDonald, Beal & Ayers, 1987/1988; Signer, 1982; & Travis. 
1984). The advantages of computer assisted testing include: (a) 
flexibility of administration time, (b) efficiency, (c) test security, (d) 
clerical processing power, and (e) reduction in test construction time 
(Lippey, 1973; Reckase, 1986; Signer, 1982). 

Recommendations support the need for the continual integration 
of assessment into classroom instruction (Bright, 1988; Bunderson, 
Inouye & Olsen, 1988; Fisher, Berliner, Filbey, Marliave, Cahen and 
Dishaw, 1980; & The National Council of Teachers of Mathematics. 1989). 
Moreover, the use of diagnostic data by teachers has been positively 
correlated with student achievement (Bright, 1987; Lamon & Bright, 
1987). 

Evaluation studies of computer use in programs for students in 
danger of dropping out of high school are few (Polin, 1989). Guerrero 
and Swan (1988) investigated the efficacy of computer assisted 
instruction (CAI) with older, at-risk students in New York City. The 
researchers reported improvement in student attitudes and pride in 
their work. Johnson & Mihal (1973) found that black elementary 
students improved their performance with computerized tests. They 
hypothesized that anxiety increases when tests are admin isiered by 
individuals representing more advantaged populations. 

Project Description 
The topics of the MATH-R-US diagnostic programs and 
accompanying practice sheets are: (a) numeration, (b) whole numbers, 
(c) fractions, (d) decimals, (e) percent , and (0 measurement. They 
were written to test every required high school general mathematics 
objective as described in the city's curriculum guide. The project was 
used for an entire school year by a class in an urban high school. The 
school serves an at-risk predominantly black population with a high 
rate of absenteeism. 

The tests accept generative responses, rather than multiple choice 
answers. If a student's answer is correct, the next objective is tested. If 
an answer is incorrect, a second question on that objective is presented. 
If that too is incorrect, than that objective is considered missed. Upon 
completion of the test, results are shown on a computer monitor, saved 
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on a disk for later retrieval, and reproduced by a printer. Practice 
sheets, with answer keys, can then generated for the missed objectives. 

The diagnostic tests were administered once a week in the school's 
computer lab. During the other daily class meetings, students worked on 
the computer generated practice sheets. Students typically would be 
working on different objectives. 

Methods 

Evaluation was not conducted to fulfill external funding 
requirements. Instead, its purpose was to improve implementation and 
furnish descriptive data to the classroom teacher and school 
administrators. In this respect, evaluation was applied according to the 
definition of Stake (1967) which is to describe inputs, processes, and 
outcomes. 

The program's author observed the computer sessions to record 
both teacher and student suggestions. Recommendations were 
incorporated into the programs and tested at the next weekly computer 
class. Weekly computer test results were kept by the teacher. For each 
student he noted when a test was taken, which objectives were missed, 
and the objective that was last attempted on the test. The teacher decided 
which tests to give to each student and whether or not they should be 
repeated. These records were used to assess the students' achievement in 
mathematics. 

A graduate student attended all the computer sessions to record on 
and off task behaviors. To ascertain student attitude toward their 
instruction, an attitude survey was administered at the end of the year. 
The attitude instrument was based on the Attitude Toward Instructional 
Setting Survey developed by Suydam (1974). All items are scored on a 
five point Likert scale (strongly agree, agree, neutral, disagree and 
strongly disagree). 

Data Source 

The students were part of a special program in an urban high 
school serving predominantly black youth. They were targeted as the 
most at-risk of dropping out before high school graduation. They had a 
history of high absenteeism and academic setbacks. 

Close to thirty students comprised the targeted group. However, 
the actual group member changed sharply throughout the year because 
of: (a) erratic attendance, (b) students dropping out of school, (c) 
students transferring to other schools, and (d) new students being 
admitted to the class. For this reason, it was not feasible to apply 
comparative testing procedures. 

Results 

One finding of the study was intense student generated 
competition. When students completed a test with 100% accuracy, a 
graphic of a hamburger appeared on the screen. The students turned 
this feature into a contest. Upon entering the computer lab, they raced 
to see who could get the most hamburgers in that hour. Another 
observation was the absence of off task behaviors. As a result of the 
student competition for hamburger pictures, students infrequently (less 
than 5% of observed computer class time) engaged in off task behavior. 

An examination of the attitude data was conducted to look for 
aggregate responses and to determine if there were differences 
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according to gender. Only students who attended at least two consecutive 
months of classes completed the survey. Fifteen students met this 
criteria. Although both males and females responded positively to the 
components surveyed overall, an inspection of the responses disclosed 
differences by gender. Results showed that the males tended to 
distribute their responses across the five responses, while the females 
clustered their responses at the extremes. 

Means for each item were computed by gender. In order to 
compare the means for each statement, many of the survey items were 
reworded for positive phrasing. Mean scores ranging from 1 to 1.49 
were interpreted to denote strongly agree; 1.50 to 2.49 agree; 2.50 to 3.49 
undecided; 3.50 to 4.49 disagree; and 4.50 to 5.00 strongly disagree. 
Attitude survey responses that reflected personal concerns (fear of 
operating the computer, receiving attention, and opinion of the teacher) 
were more positive and stronger among the females. This was also true 
for items that described the classroom environment as conducive to 
learning. Table 1 contains a list of the attitude statements that were 
rated differently by the nine males and six females. While both males 
and females expressed positive attitudes about the course components, it 
is interesting that the female responses reflected more confidence in 
one's own abilities. 

An observation that occurred with increasing frequency was 
student insistence that answers were correct, when the computer 
indicated otherwise. As a result, the students were very eager to prove 
themselves to their teacher. This observation was surprising since 
students, especially low achieving students, are not inclined to review 
examples they are told are incorrectly solved. 

Achievement in mathematics was studied by examining student 
performance on the diagnostic tests during the year. The unstable 
student population demanded that this assessment look at individual 
student performance on repeated tests. The data in Table 5 reports test 
results for students who took the same topic test more than once. Only 17 
of 48 students met this criterion. While many other students took several 
computer tests, they did not attend often enough to have engaged in the 
same test more than once. 

An inspection of the computer math test scores indicated a 
consistent improvement on retesting of a topic, with a 25% average 
increase over an earlier test. Of the 43 retests, only 3 did not show an 
improvement. The 23 perfect scores are noteworthy, in light of the 
difficulty of many of the test items. Table 2 lists the more difficult 
objectives for the topic tests. 

Educational Implications 
This study supports the findings of Becker (1988) and Guerrero 
and Swan (1988). At-risk high school minority students can profit from 
computer assisted instruction with increased motivation, self- 
confidence, and self-discipline. Even though the students were being 
asked to solve 100% of the topic objectives with 100% accuracy (without 
guessing), they did not feel that too much work was required for the 
course. 

It is noteworthy that the girls were more definitive and self- 
assured than the boys. Girls have historically been described as more 
computer anxious than boys. Nevertheless, unlike the boys, the girls 
were undivided in their agreement that they were confident they could 
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operate the computer. They viewed the computer as having a positive 
effect on their learning and expressed a sense of active enjoyment. 
These results are considerably different from those of Campbell and 
Perry (1988). Their results found more minority stereotyping of 
computers as a male domain. The majority of students in the Campbell 
and Prny study had completed a computer course. In contrast, those in 
this study were at-risk students who used the computer as a tool rather 
than as the object of instruction. An explanation of the results of this 
project raises the possibility that older at-risk, black girls are not as 
computer anxious as girls have been reported. 

The high risk black high school girls in this study seemed to 
exhibit higher self-confidence toward computers than that found in 
other studies that examined gender. Recognizing that generalizations 
from small samples are dangerous, the results of this investigation solely 
suggest that research is needed to investigate computer and mathematics 
anxiety of black teenage girls. Specificly, studies should examine the 
mathematics and computer anxiety of black, at-risk, high school girls, 
when the computer is used as a tool, rather than as the object of 
instruction. 

Previous research has found a positive correlation of computer 
access and use with general school achievement (Becker (1987); 
Blaschke (1986); Gannon (1986); Krulik & Krulik (1987). More recently, 
the availability of computers in middle- and upper-income homes has 
exacerbated the school equity/access problem for the black at-risk 
student. In light of the results of this project and the equity issue, 
technology affirmative action seems necessary so that low achievers will 
have more opportunities, rather than fewer, to raise their levels of 
achievement. 
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Table 1 

Varying Mean Responses to the Attitude Survey bv Gender 



ATTITUDE 
CATEGORY 



ATTITUDE STAT 



ST 



MEAN RATING 
M F 



Computer 
Confidence 

Computer 
Value 



I was afraid I couldn't operate 
the computer. 

I feel the computer was a 
waste of time. 

Using the computer was 
boring. 



D 



U 



SD 
SD 
D 



Confidence 
could do the 
work. 



Too much work was required 
for this class. 

The slow worker didn't have 
a chance in this class. 



D 



U 



SD 



D 



Feel 

Important 



If a student did not learn in 
this class, it was his/her own 
fault. 

Using the computer made me 
feel that I am important. 



U 



U 



D 



D 



Computer de- 
personalizes 
instruction. 

Enjoyed 
Computer 



No oae paid any attention to 
me when we used the 
computer. 

Time seemed to fly when we 
used the computers. 



D 



U 



SD 



SA= Strongly Agree; 
D = Disagree; 



A ~ Agree; 
SD = Strongly 



U = Undecided; 
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Table 2 




Descriptors of Difficult Test Items 


TOPIC 


OBJECTIVE 


Numeration 


Estimating the square root to nearest tenth 


Fractions 


Finding a fractional part in a word problem 

Adding unlike denominators 

Subtracting unlike denominators 

Dividing mixed numbers 

Solving ratio and proportion problems 


Percent 


Finding percent when the percent is a decimal 
Finding percent in a word problem 
Finding a number when the percent is known 
Finding percent of discount, given original and 
sale prices 

Finding the cost given the percent of discount 


Measurement 


Doing two-step conversions, within a system 
Applying knowledge of measurement to word 
problems 
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